Three-dimensional display device, three-dimensional image processing device, and three-dimensional display method

ABSTRACT

A three-dimensional display device for displaying a three-dimensional image, including: a gaze point obtaining unit which obtains a position of a gaze point of a viewer; a fusional area determination unit which determines a fusional area where binocular fusion is allowed, based on the obtained position of the gaze point; a correction unit which corrects the three-dimensional image so as to suppress display of an object that is included in the three-dimensional image outside the fusional area; and a display unit which displays the corrected three-dimensional image.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation application of PCT International Application No.PCT/JP2013/005666 filed on Sep. 25, 2013, designating the United Statesof America, which is based on and claims priority of Japanese PatentApplication No. 2012-224565 filed on Oct. 9, 2012. The entiredisclosures of the above-identified applications, including thespecifications, drawings and claims are incorporated herein by referencein their entirety

FIELD

One or more exemplary embodiments disclosed herein relate generally totechnology for displaying or processing a three-dimensional medicalimage.

BACKGROUND

Changing a viewpoint (e.g., rotation or zoom) of a three-dimensional,image (e.g., a left-eye image and a right-eye image) changes apositional relationship in the depth direction between a plurality ofobjects included in the three-dimensional image or between portions ofan object. As a result, for example, an object located on a far side ishidden by an object on a near side, reducing the visibility of thethree-dimensional image.

In response, Patent Literature (PTL) 1 discloses a method for adjustingtransparency of an object included in a three-dimensional image inaccordance with a depth of the object. This can display an objectlocated on a far side through an object located on a near side.

Moreover, PTL 2 discloses a method for maintaining stereoscopic effectsby adjusting an amount of a disparity between a left-eye image and aright-eye image in accordance with a focal length upon zooming athree-dimensional image.

CITATION LIST Patent Literature

[PTL 1] Japanese Unexamined Patent Application Publication No.2001-84409

[PTL 2] Japanese Unexamined Patent Application Publication No.2011-248693

SUMMARY Technical Problem

In the above conventional methods, however, the visibility of thethree-dimensional image may decrease depending on a state of a viewer ora three-dimensional image.

Thus, one non-limiting and exemplary embodiment provides athree-dimensional display device which can improve visibility of athree-dimensional image, adapting to a state of a viewer or thethree-dimensional image.

Solution to Problem

In one general aspect, the techniques disclosed here feature athree-dimensional display device for displaying a three-dimensionalimage, including: a gaze point obtaining unit configured to obtain aposition of a gaze point of a viewer; a fusional area determination unitconfigured to determine a fusional area where binocular fusion isallowed, based on the obtained position of the gaze point; a correctionunit configured to correct the three-dimensional image so as to suppressdisplay of an object which is included in the three-dimensional imageoutside the fusional area; and a display unit configured to display thecorrected three-dimensional image.

These general and specific aspects may be implemented using a system, amethod, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, methods, integrated circuits, computer programs, orcomputer-readable recording media such as CD-ROM.

Additional benefits and advantages of the disclosed embodiments will beapparent from the Specification and Drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the Specification and Drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

Additional benefits and advantages of the disclosed embodiments will beapparent from the Specification and Drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the Specification and Drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

Advantageous Effects

According to a three-dimensional display device of one or more exemplaryembodiments or features disclosed herein, visibility of athree-dimensional image can be improved, adapting to a state of a vieweror the three-dimensional image.

BRIEF DESCRIPTION OF DRAWINGS

These and other advantages and features will become apparent from thefollowing description thereof taken in conjunction with the accompanyingDrawings, by way of non-limiting examples of embodiments disclosedherein.

FIG. 1 shows schematic views illustrating examples of an image of ablood vessel displayed three-dimensionally.

FIG. 2 is a block diagram of a functional configuration of athree-dimensional display device according to an embodiment 1.

FIG. 3 is a flowchart illustrating processing operation of thethree-dimensional display device according to the embodiment 1.

FIG. 4 shows schematic views for illustrating processing operation ofthe three-dimensional display device according to the embodiment 1.

FIG. 5 shows diagrams illustrating examples of the three-dimensionalimage displayed by the three-dimensional display device according to theembodiment 1.

FIG. 6 is a block diagram of a functional configuration of athree-dimensional display device according to an embodiment 2.

FIG. 7 is a diagram showing an example of image information in theembodiment 2.

FIG. 8A is a schematic view illustrating an example of a sensoraccording to the embodiment 2.

FIG. 8B is a schematic view illustrating another example of the sensoraccording to the embodiment 2.

FIG. 9 is a schematic view illustrating another example of the sensoraccording to the embodiment 2.

FIG. 10 is a schematic view illustrating another example of the sensoraccording to the embodiment 2.

FIG. 11 is a schematic view illustrating an example of a coordinatesystem in the embodiment 2.

FIG. 12 is a diagram showing an example of fusional area information inthe embodiment 2.

FIG. 13 is a flowchart illustrating processing operation of thethree-dimensional display device according to the Embodiment 2

FIG. 14 is a flowchart illustrating processing operation of a viewpointchanging unit according to the embodiment 2.

FIG. 15 shows schematic views illustrating an example of coordinatetransformation according to the embodiment 2.

FIG. 16 is a flowchart illustrating processing operation of a correctionprocess determination unit according to the embodiment 2.

FIG. 17 is a flowchart illustrating processing operation of an imageprocessing unit according to the embodiment 2.

FIG. 18 shows diagrams illustrating an example of a three-dimensionalimage displayed by the three-dimensional display device according to theembodiment 2.

FIG. 19 shows schematic views for illustrating viewpoint changeaccording to a variation of the embodiment 2

FIG. 20 is a block diagram of a functional configuration of athree-dimensional display device according to an embodiment 3.

FIG. 21 is a diagram showing an example of image information in theembodiment 3.

FIG. 22 is a schematic view illustrating an example of a method todetermine a branch number according to the embodiment 3.

FIG. 23 is a flowchart illustrating processing operation of thethree-dimensional display device according to the embodiment 3.

FIG. 24 is a block diagram of a detailed functional configuration of ablood vessel connection information extraction unit according to theembodiment 3.

FIG. 25 is a block diagram of a detailed functional configuration of ablood vessel importance calculation unit according to the embodiment 3.

FIG. 26 is a flowchart illustrating processing operation of the bloodvessel importance calculation unit according to the embodiment 3.

FIG. 27A is a diagram showing an example of a score translation table inthe embodiment 3.

FIG. 27B is a diagram showing an example of the score translation tablein the embodiment 3.

FIG. 27C is a diagram showing an example of the score translation tablein the embodiment 3.

FIG. 28 is a block diagram of a detailed functional configuration of asegmentation unit according to the embodiment 3.

FIG. 29 is a diagram showing an example of a segmentation table in theembodiment 3.

FIG. 30 is a block diagram of a functional configuration of athree-dimensional display device according to an embodiment 4.

FIG. 31 is a block diagram of a detailed functional configuration of ablood vessel importance calculation unit according to the embodiment 4.

FIG. 32 is a diagram showing an example of instrument information in theembodiment 4.

FIG. 33 is a diagram showing an example of image information in theembodiment 4.

FIG. 34A is a diagram showing an example of a score translation table inthe embodiment 4.

FIG. 34B is a diagram showing an example of the score translation tablein the embodiment 4.

FIG. 34C is a diagram showing an example of the score translation tablein the embodiment 4.

FIG. 34D is a diagram showing an example of the score translation tablein the embodiment 4.

FIG. 34E is a diagram showing an example of the score translation tablein the embodiment 4.

FIG. 35 is a block diagram of a functional configuration of athree-dimensional display device according to an embodiment 5.

FIG. 36 is a flowchart illustrating processing operation of thethree-dimensional display device according to the embodiment 5.

FIG. 37 is a block diagram of a functional configuration of athree-dimensional image processing device according to an embodiment 6.

DESCRIPTION OF EMBODIMENTS

(Underlying Knowledge Forming Basis of the Present Disclosure)

When a displayed three-dimensional image changes or a position of apoint of gaze of a viewer changes, a relationship changes between anobject and a portion of the object (hereinafter, these are collectivelyreferred to as a “near side portion”) which are displayed projectingtoward a side closer to the viewer than an object and a portion of theobject which a viewer is gazing (hereinafter, these are collectivelyreferred to as a “gaze portion”) and an object and a portion of theobject displayed receding into a side (hereinafter, these arecollectively referred to as a “far side portion”) farther away from theviewer than the gaze portion is. At this time, visibility of the gazeportion decreases due to positions or patterns of the near side portionand the far side portion. For example, when the near side portionoverlaps with the gaze portion in the depth direction, the gaze portionends up being hidden by the near side portion.

In response, according to the method disclosed in PTL 1, visibility of agaze portion can be improved by adjusting transparency of a near sideportion. The method of PTL 1, however, requires a viewer to manuallyadjust transparency of an object or a portion of the object in athree-dimensional image. In other words, the method of PTL 1 requires aviewer to manually adjust the transparency of the object or the portionthereof every time a state of the viewer or the three-dimensional imagechanges.

Moreover, in the method of PTL 2, an amount of a disparity can beautomatically adjusted in accordance with a focal length whereasvisibility of a gaze portion cannot be improved.

Herein, an example of a state where the gaze portion is hidden by thenear side portion in the three-dimensional image will be described indetail, with reference to a three-dimensional image of a blood vessel(hereinafter, simply referred to as an “image of a blood vessel”).

FIG. 1 shows examples of an image of a blood vessel displayedthree-dimensionally.

In (a) of FIG. 1, a blood vessel is displayed along a display screen. Asthe image of the blood vessel displayed in (a) of FIG. 1 is rotated, theimage of the blood vessel is displayed, for example, as shown in (b) ofFIG. 1. Here, the image of the blood vessel is rotated about, as therotation axis, an axis of the vertical direction passing through thecenter of the screen.

The blood vessel displayed in (a) of FIG. 1 has two curved portions onthe left and right sides of the screen. A viewer performs an operationfor rotating the image of the blood vessel to see the curved portion onthe left side from a different angle. In the example of FIG. 1, theimage is rotated such that an object on the left side of the screenmoves to the front of the screen and an object on the right side of thescreen moves behind the screen.

At this time, as (b) of FIG. 1 shows, the curved portion (the gazeportion) which the viewer desires to view is displayed at athree-dimensional position closer to the viewer than the screen is.However, the portion displayed on the left of the gaze portion in (a) ofFIG. 1 is displayed even closer to the viewer than the gaze portion is.As a result, the portion (the near side portion), which is displayedcloser to the viewer than the gaze portion is, ends up hiding the gazeportion.

Furthermore, the blood vessel disposed along with the depth directioncauses a large difference in depth between an end portion of the bloodvessel on the front side and an end portion on the far side. This causesthe object other than portions thereof near the gaze portion to appeardouble to the viewer.

The depth of a three-dimensional image is represented by a disparitybetween a left-eye image and a right-eye image. Thus, athree-dimensional image having portions which are significantlydifferent in depth includes areas which are significantly different indisparity.

When seeing a three-dimensional image, a person adjusts an angle betweenthe left and right eyes (an angle of convergence) to the disparity. As aresult, the person can overlap two images obtained from the left eye andthe right eye to obtain an image, thereby viewing an object in astereoscopic manner. Overlapping two images obtained from both eyes toobtain an image as such is referred to as binocular fusion or, simply,fusion.

For example, for a three-dimensional image having two objects which aresignificantly different from each other in disparity, if an angle ofconvergence is adjusted to the disparity of one of the two objects, theangle of convergence does not conform to the disparity of the other ofthe two objects. As a result, the object that has the disparity to whichthe angle of convergence does not conform ends up causing double vision(diplopia). In other words, double vision outside the fusional area isundesirably produced.

In (a) of FIG. 1, there are small changes in depth between the gazeportion and the other portions. Thus, differences in disparity in thethree-dimensional image are also small. Therefore, adjusting the angleof convergence so as to conform to the disparity of the gaze portionprevents double vision from occurring.

On the other hand, in (b) of FIG. 1, a three-dimensional position in thedepth direction at which the blood vessel is displayed is significantlydifferent depending on a portion of the blood vessel. Thus, for example,a disparity at a near side portion is greater than a disparity at thegaze portion, wherein, it is known that if the angle of convergence isadjusted conforming to the disparity at the gaze portion, a disparity(i.e., depth) where the fusion is allowed by the angle of convergencehas a certain width rather than being limited to the disparity (i.e.,depth) at the gaze portion.

An area where binocular fusion is allowed will be referred to asfusional area (fusional limits of depth perception). A fusional area hasa viewing angle of a few minutes at the center of a field of view, whichis extremely narrower than that at the periphery (Panum's fusionalarea). Moreover, the fusional area reduces as the gaze portion islocated on a side closer to the viewer in the depth direction, andincreases as the gaze portion is located on a side farther away from theviewer. Thus, double vision is more likely to be caused as closer to theviewer the gaze portion is located.

Thus, a three-dimensional display device according to an exemplaryembodiment disclosed herein is a three-dimensional display device fordisplaying a three-dimensional image, including: a gaze point obtainingunit configured to obtain a position of a gaze point of a viewer; afusional area determination unit configured to determine a fusional areawhere binocular fusion is allowed, based on the obtained position of thegaze point; a correction unit configured to correct thethree-dimensional image so as to suppress display of an object which isincluded in the three-dimensional image outside the fusional area; and adisplay unit configured to display the corrected three-dimensionalimage.

According to the above configuration, the three-dimensional image can becorrected so as to suppress display of objects that are included in thethree-dimensional image outside the fusional area. This can thereforesuppress visual effects which are caused by double vision producedoutside the fusional area. As a result, the visibility of an objectwhich the viewer is gazing in the three-dimensional image can beimproved.

Furthermore, according to the above configuration, the three-dimensionalimage can be corrected using a fusional area that is automaticallydetermined in accordance with a position of a point of gaze. Thus, theviewer is not required to designate an object the display of which is tobe suppressed, thereby improving viewer convenience as well.

For example, the correction unit may correct the three-dimensional imageby removing an object which is included in the three-dimensional imageand located on a side closer to the viewer than the fusional area is.

According to the above configuration, objects located on the nearer sidethan the fusional area can be removed from the three-dimensional image.Thus, double vision in an area on the nearer side than the fusional areacan be prevented from occurring. Furthermore, an object which the vieweris gazing can be prevented from being hidden by another object. As aresult, further improvement in the visibility of the object which theviewer is gazing in the three-dimensional image is possible.

For example, the correction unit may correct the three-dimensional imageby blurring an object which is included in the three-dimensional imageand located on a side farther away from the viewer than the fusionalarea is.

According to the above configuration, an object located on the fartherside than the fusional area can be blurred. This can therefore suppressvisual effects which are caused by double vision produced on the fartherside than the fusional area.

For example, the three-dimensional display device may further include aviewpoint changing unit configured to change a viewpoint of thethree-dimensional image so that a display position of an object which isincluded in the three-dimensional image and located at the position ofthe gaze point does not change in a depth direction, wherein thecorrection unit may correct the three-dimensional image the viewpoint ofwhich has been changed.

According to the above configuration, the viewpoint of thethree-dimensional image can be changed so that the display position ofthe object displayed at the position of point of gaze does not change inthe depth direction. Thus, the viewer can continue gazing at the sameobject without changing the angle of convergence. Thus, load imposed onthe viewer can be reduced.

For example, changing the viewpoint may be a process of rotating thethree-dimensional image about the position of the gaze point.

According to the above configuration, the rotation process about theposition of the point of gaze can be performed on the three-dimensionalimage. As a result, the viewpoint of the three-dimensional image can bechanged so that the display position of the object displayed at theposition of point of gaze does not change.

For example, the three-dimensional display device may further include afusional area information storage unit configured to store fusional areainformation which indicates positions of a plurality of gaze points in adepth direction of the three-dimensional image and a plurality offusional areas corresponding to the positions of the plurality of gazepoints in the depth direction, wherein the fusional area determinationunit may refer to the fusional area information to determine thefusional area that corresponds to the obtained position of the gazepoint.

According to the above configuration, referring to the fusional areainformation facilitates determination of a fusional area thatcorresponds to the obtained position of the point of gaze.

For example, the three-dimensional image may include a plurality ofblood vessel objects representing a plurality of blood vessels, thethree-dimensional display device further including: a blood vesselconnection information obtaining unit configured to obtain connectioninformation indicating connectivity of a blood vessel object located atthe gaze point to each of the blood vessel objects included in thethree-dimensional image; and a blood vessel importance calculation unitconfigured to calculate importance of each of the blood vessel objectsincluded in the three-dimensional image, based on the fusional area andthe connection information, wherein the correction unit may correct thethree-dimensional image so that display of a blood vessel object theimportance of which is lower is suppressed to a greater extent.

According to the above configuration, the three-dimensional image can becorrected so that display of a blood vessel object the importance ofwhich is lower is suppressed to a greater extent, the importance beingcalculated based on the fusional area and the connection information.Thus, suppression of display of a blood vessel object in accordance withthe blood vessel object located at the point of gaze is possible.

For example, the blood vessel importance calculation unit may calculate,each of the blood vessel objects, the importance of the blood vesselobject so that the blood vessel object, if included in the fusionalarea, is of higher importance than if the blood vessel object is notincluded in the fusional area.

According to the above configuration, importance of a blood vesselobject can be calculated so that the blood vessel object, if included inthe fusional area, is of higher importance than if the blood vesselobject is not included in the fusional area. Thus, suppression of bloodvessel objects that not included in the fusional area is possible.

For example, the blood vessel importance calculation unit may calculate,for each of the blood vessel object, the importance of the blood vesselobject so that the blood vessel object having a less number of bloodvessel branches to the blood vessel object located at the gaze point isof higher importance.

According to the above configuration, importance of a blood vesselobject can be calculated so that the blood vessel object having a lessnumber of blood vessel branches to the blood vessel object located atthe point of gaze is of higher importance. Thus, display of blood vesselobjects which are connected to the blood vessel object located at thepoint of gaze via a large number of branches can be suppressed, therebyimproving the visibility of the blood vessel object located at the pointof gaze.

For example, the blood vessel importance calculation unit may calculate,for each of the blood vessel objects, the importance of the blood vesselobjects so that the blood vessel object having a smaller spatialdistance to the blood vessel object located at the gaze point is ofhigher importance.

According to the above configuration, importance of a blood vesselobject can be calculated so that the blood vessel object having asmaller spatial distance to the blood vessel object located at the pointof gaze is of higher importance. Thus, display of blood vessel objectsthat have great spatial distances to the object located at the point ofgaze can be suppressed, thereby improving the visibility of the bloodvessel object located at the point of gaze.

For example, the three-dimensional image may include a plurality ofblood vessel objects representing a plurality of blood vessels, and aninstrument object representing a medical instrument which is advancedthrough at least one of the blood vessel objects, the three-dimensionaldisplay device further including an identification unit configured toidentify, among the plurality of blood vessel objects, a blood vesselobject through which the instrument object has already passed or a bloodvessel object through which the instrument object does not pass, whereinthe correction unit may correct the three-dimensional image so as tosuppress display of a blood vessel object which is located outside thefusional area and through which the instrument object has already passedor display of a blood vessel object which is located outside thefusional area and through which the instrument object does not pass.

According to the above configuration, the three-dimensional image can becorrected so as to suppress display of a blood vessel object throughwhich the instrument object has already passed or display of a bloodvessel object through which the instrument object does not pass. Thus,display of a blood vessel object through which the instrument object islikely to be advanced can be prioritized, thereby displaying a usefulthree-dimensional image.

Moreover, a three-dimensional image processing device according to anexemplary embodiment disclosed herein is a three-dimensional imageprocessing device for processing a three-dimensional image including aplurality of blood vessel objects representing a plurality of bloodvessels, and an instrument object representing a medical instrumentwhich is advanced through at least one of the blood vessel objects, thethree-dimensional image processing device including: a gaze pointobtaining unit configured to obtain a position of a gaze point of aviewer; a fusional area determination unit configured to determine afusional area where binocular fusion is allowed, based on the obtainedposition of the gaze point; an identification unit configured toidentify, among the plurality of blood vessel objects, a blood vesselobject through which the instrument object has already passed or a bloodvessel object through which the instrument object does not pass; and acorrection unit configured to correct the three-dimensional image so asto suppress display of a blood vessel object which is located outsidethe fusional area and through which the instrument object has alreadypassed or display of a blood vessel object which is located outside thefusional area and through which the instrument object does not pass.

These general and specific aspects may be implemented using a system, amethod, an integrated circuit, a computer program, or acomputer-readable recording medium such as a CD-ROM, or any combinationof systems, methods, integrated circuits, computer programs, orcomputer-readable recording media such as CD-ROM.

Hereinafter, certain exemplary embodiments will be described in greaterdetail, with reference to the accompanying drawings.

It should be noted that the non-limiting embodiments described below aregeneral and specific illustration. Values, shapes, materials,components, disposition or a form of connection between the components,steps, and the order of the steps described in the following embodimentsare merely illustrative, and are not intended to limit the appendedclaims. Moreover, among components of the below embodiments, componentsnot set forth in the independent claims indicating the top level conceptof the present disclosure will be described as optional components.

In the following, unnecessarily detailed description may be omitted. Forexample, detailed description of well-known matters or descriptionpreviously set forth with respect to components that are substantiallythe same may be omitted. This is to avoid unnecessarily redundancy inthe description below and for facilitating an understanding by thoseskilled in the art.

Embodiment 1

<Configuration>

FIG. 2 is a block diagram of a functional configuration of athree-dimensional display device 10 according to an embodiment 1. Thethree-dimensional display device 10 displays a three-dimensional image.In other words, the three-dimensional display device 10 displays animage in a stereoscopic manner. Specifically, the three-dimensionaldisplay device 10 displays a three-dimensional image by a glasses basedstereoscopic display method. The glasses based stereoscopic displaymethod is for displaying a left-eye image and a right-eye image, whichhave the disparity therebetween, to a viewer wearing glasses (such asLCD shutter glasses and polarized glasses). Also for example, thethree-dimensional display device 10 may display a three-dimensionalimage by autostereoscopy. The autostereoscopy is a stereoscopic displaymethod without the use of glasses (such as parallax barrier andLenticular lens technologies).

As FIG. 2 shows, the three-dimensional display device 10 includes a gazepoint obtaining unit 11, a fusional area determination unit 12, acorrection unit 13, and a display unit 14.

The gaze point obtaining unit 11 obtains a position of a point of gazeof a viewer. Specifically, the gaze point obtaining unit 11, forexample, detects a three-dimensional position of a point of gaze, basedon electrooculograms of a viewer or an image of the eyes. It should benoted that the gaze point obtaining unit 11 may not detect a position ofthe point of gaze. For example, the gaze point obtaining unit 11 mayobtain a position of the point of gaze by obtaining informationindicative of the position of the point of gaze from a sensor or thelike provided external to the three-dimensional display device 10.

The fusional area determination unit 12 determines a fusional area wherebinocular fusion is allowed, based on the obtained position of the pointof gaze. Specifically, the fusional area determination unit 12determines a fusional area corresponding to the obtained position of thepoint of gaze by, for example, referring to fusional area informationindicating positions of a plurality of points of gaze in the depthdirection and a plurality of fusional areas corresponding to therespective positions of the plurality of points of gaze in the depthdirection. Also for example, the fusional area determination unit 12 maydetermine a fusional area by a mathematical formula for calculating thefusional area from the position of the point of gaze.

The correction unit 13 corrects the three-dimensional image so as tosuppress display of an object that is included in the three-dimensionalimage outside the fusional area. Suppressing the display of an objectmeans lowering a display level of the object. For example, the displayof the object can be controlled by increasing the transparency of theobject or reducing the definition of the object. It should be noted thatcontrolling the display of an object includes hiding (transparentizing)the object.

Specifically, the correction unit 13 corrects the three-dimensionalimage by, for example, removing an object which is included in thethree-dimensional image and located on a side closer to the viewer(hereinafter, referred to as a nearer side) than the fusional area is.Also for example, the correction unit 13 may correct thethree-dimensional image by blurring an object which is included in thethree-dimensional image and located on a side farther away from theviewer (hereinafter, referred to as a farther side) than the fusionalarea is.

More specifically, the correction unit 13 renders a left-eye image and aright-eye image, using only three-dimensional models located within anarea corresponding to the fusional area in a virtual three-dimensionalspace where, for example, a plurality of three-dimensional modelsindicating a plurality of objects included in the three-dimensionalimage are disposed. Also for example, the correction unit 13 mayincrease transparency of three-dimensional models outside the areacorresponding to the fusional area, and then render a left-eye image anda right-eye image.

The display unit 14 displays the corrected three-dimensional image.Specifically, the display unit 14, for example, alternately displays theleft-eye image and the right-eye image on the screen, thereby displayingthe corrected three-dimensional image. Also for example, the displayunit 14, for example, displays the left-eye image using, among aplurality of pixels arrayed in matrix on the screen, left pixels whichcan be seen by the left eye, and displays the right-eye image usingright pixels which can be seen by the right eye.

It should be noted that the display unit 14 may not have a screen. Inthis case, the display unit 14 may display the three-dimensional imagevia a display device external to the three-dimensional display device10.

Next, various operations of the three-dimensional display device 10configured as set forth above will be described.

<Operation>

FIG. 3 is a flowchart illustrating processing operation of thethree-dimensional display device 10 according to the embodiment 1.

First, the gaze point obtaining unit 11 obtains a position of a point ofgaze of the viewer (S11). Subsequently, the fusional area determinationunit 12 determines a fusional area, based on the position of the pointof gaze (S12). Then, the correction unit 13 corrects thethree-dimensional image so as to suppress display of an object that isincluded in the three-dimensional image outside the fusional area (S13).Last, the display unit 14 displays the corrected three-dimensional image(S14).

The processing operation of the three-dimensional display device 10 willbe described in more detail, with reference to FIGS. 4 and 5.

FIG. 4 shows schematic views for illustrating the processing operationof the three-dimensional display device 10 according to theembodiment 1. Specifically, (a) of FIG. 4 is a diagram showing apositional relationship between objects 40, 41, and 42 included in thethree-dimensional image. Part (b) of FIG. 4 is an explanatory diagram ofdouble vision.

FIG. 5 shows diagrams illustrating examples of the three-dimensionalimage which is displayed by the three-dimensional display device 10according to the embodiment 1. Specifically, (a) of FIG. 5 shows anuncorrected three-dimensional image, and (b) of FIG. 5 shows a correctedthree-dimensional image.

As illustrated in (a) of FIG. 4, the object 41 is displayed on a nearerside than the object 40, and the object 42 is displayed on a fartherside than the object 40. As (b) of FIG. 4 illustrates, a disparity D1 ofthe object 40 is smaller than a disparity D2 of the object 41 andgreater than a disparity of the object 42.

Here, suppose that the position of the point of gaze is a displayposition of the object 40, the fusional area is a neighboring area ofthe object 40. At this time, the object 41 is located outside (thenearer side than) the fusional area. In other words, the angle ofconvergence of the viewer does not conform to the disparity of theobject 41. Thus, double vision 41 d is caused. The object 42 is alsolocated outside (the farther side than) the fusional area. In otherwords, the angle of convergence of the viewer does not conform to thedisparity of the object 42. Thus, double vision 42 d is caused. In otherwords, the viewer perceives the three-dimensional image as shown in (a)of FIG. 5.

In this case, particularly, the double vision 41 d on the nearer sidethan the object 40, which is a gaze object, reduces the visibility ofthe object 40. Thus, for example, by removing the object 41 from thethree-dimensional image, the correction unit 13 corrects thethree-dimensional image so as to suppress the display of the object 41.As a result, as (b) of FIG. 5 shows, the double vision 41 d on thenearer side than the object 40 is removed, improving the visibility ofthe object 40.

Moreover, for example, if the objects 40 and 42 have a similar pattern,the double vision 42 d on the farther side than the object 40 alsoreduces the visibility of the object 40. Thus, for example, by blurringthe object 42 in the three-dimensional image, the correction unit 13corrects the three-dimensional image so as to suppress the display ofthe object 42. As a result, the object 40 is highlighted greater thanthe double vision 42 d, improving the visibility of the object 40.

<Effects>

As described above, according to the present embodiment, thethree-dimensional display device 10 can correct the three-dimensionalimage so as to suppress display of objects that are included in thethree-dimensional image outside the fusional area. Thus, thethree-dimensional display device 10 can suppress visual effects whichare caused by double vision produced outside the fusional area. As aresult, the three-dimensional display device 10 can improve thevisibility of an object which the viewer is gazing in thethree-dimensional image.

Furthermore, according to the present embodiment, the three-dimensionaldisplay device 10 can correct the three-dimensional image, using afusional area that is automatically determined in accordance with aposition of a point of gaze. Thus, the viewer is not required todesignate an object the display of which is to be suppressed. Thethree-dimensional display device 10 can thus improve viewer convenienceas well.

Embodiment 2

Next, an embodiment 2 will be described in detail. In the presentembodiment, description will be given where a viewpoint of a displayedthree-dimensional image is changed.

<Configuration>

FIG. 6 is a block diagram of a functional configuration of athree-dimensional display device 100 according to the embodiment 2.

The three-dimensional display device 100 includes an image informationstorage unit 101, a viewpoint input unit 102, a viewpoint changing unit103, a depth calculation unit 104, a sensor 105, a gaze point detectionunit 106, an image transformation center determination unit 107, afusional area information storage unit 108, a fusional areadetermination unit 109, a segmentation unit 110, a correction processdetermination unit 111, an image processing unit 112, and a display unit113.

The image information storage unit 101 is, for example, a hard diskdrive or a semiconductor memory. The image information storage unit 101stores image information 101 a on a three-dimensional image which isdisplayed. Specifically, the image information storage unit 101, forexample, stores three-dimensional models of objects included in thethree-dimensional image, as the image information 101 a.

In the present embodiment, the image information storage unit 101 storesinformation on blood vessel objects, each representing a blood vessel,as the image information 101 a. The image information 101 a includescoordinates and display colors of points on a blood vessel object, forexample, as shown in FIG. 7.

The viewpoint input unit 102 is an interface which inputs operation forchanging a viewpoint (e.g., rotation or zoom) of a display image.Examples of the viewpoint input unit 102 include a mouse, a remotecontroller, and a gesture detection device. Herein, viewpoint changerefers to a process of changing a viewpoint and a viewing direction (aposition and a direction of a virtual camera) with respect to an objectincluded in a three-dimensional image. For example, changing a viewpointof a three-dimensional image such that the viewpoint approaches anobject enlarges the object.

Using the image information 101 a stored in the image informationstorage unit 101, the viewpoint changing unit 103 changes a viewpoint ofthe three-dimensional image, based on the operation for changing theviewpoint received by the viewpoint input unit 102. At this time, theviewpoint changing unit 103 changes the viewpoint of thethree-dimensional image so that a display position of an object that isincluded in the three-dimensional image and displayed at a position ofthe point of gaze does not change in the depth direction.

The viewpoint change, is, for example, a process of rotating thethree-dimensional image about the position of the point of gaze.Specifically, the viewpoint changing unit 103 changes the viewpoint ofthe three-dimensional image, using a central position, which isdetermined by the image transformation center determination unit 107, asa central position about which the viewpoint is to be changed. Forexample, the viewpoint changing unit 103 rotates the three-dimensionalimage about an axis in the vertical direction passing through thecentral position determined by the image transformation centerdetermination unit 107.

The depth calculation unit 104 calculates a depth of the object includedin the three-dimensional image which has undergone the viewpoint changeby the viewpoint changing unit 103.

The sensor 105 senses eye movement of a viewer viewing thethree-dimensional image. The sensor 105 is a visible light camera or aninfrared camera which captures images of the viewer's eyes, for example,as shown in FIG. 8A or 8B. Also for example, the sensor 105 may beelectrooculogram measuring means or myopotential measuring means whichrecords changes in potential associated with the eye movement fromelectrodes in contact with a skin as illustrated in FIG. 9. Also forexample, the sensor 105 may be a search coil which records changes inpotential associated with the eye movement and movement of the iris ofthe eye or movement of the lens, from a coil in contact with the surfaceof an eyeball as illustrated in FIG. 10.

The gaze point detection unit 106 calculates a position (athree-dimensional position) of a point of gaze of the viewer, usinginformation on the viewer's eye movement sensed by the sensor 105. Atthis time, a coordinate system for representing the three-dimensionalposition is a three-dimensional display coordinate system, for example,as shown in FIG. 11. In the three-dimensional display coordinate systemof FIG. 11, origin is at the center of a screen surface, the horizontaldirection on a screen plane is the X axis, the vertical direction on thescreen plane is the Y axis, and a direction perpendicular to the screenplane is the Z axis.

In other words, in the present embodiment, the position of the point ofgaze of the viewer is obtained, using the sensor 105 and the gaze pointdetection unit 106. It should be noted that the sensor 105 and the gazepoint detection unit 106 may not be included in the three-dimensionaldisplay device 100. In this case, the three-dimensional display device100 may obtain the position of the point of gaze of the viewer which isdetected by the sensor 105 and the gaze point detection unit 106.

The image transformation center determination unit 107 determines theposition of the point of gaze of the viewer detected by the gaze pointdetection unit 106 as a central position about which the viewpoint ofthe currently displayed three-dimensional image is to be changed(rotation or zoom).

The fusional area information storage unit 108 stores fusional areainformation 108 a for obtaining a fusional area according to the depthof the point of gaze. The fusional area information 108 a indicates thedepth of the point of gaze, and a near side fusion limit and a far sidefusion limit which correspond to the depth, for example, as shown inFIG. 12. In other words, the fusional area information 108 a indicatespositions of a plurality of points of gaze in the depth direction, and aplurality of fusional areas corresponding to the plurality of points ofgaze in the depth direction.

The fusional area herein refers to an area where binocular fusion isallowed at one time, without moving the eyes (i.e., without changing theangle of convergence). The fusional area is known as Panum's fusionalarea.

The fusional area determination unit 109 refers to the fusional areainformation 108 a to determine a fusional area which corresponds to theposition of the point of gaze detected by the gaze point detection unit106. Specifically, the fusional area determination unit 109, forexample, refers to the fusional area information 108 a of FIG. 12 toobtain a near side fusion limit and a far side fusion limit whichcorrespond to the depth of the point of gaze. Then, the fusional areadetermination unit 109 determines, as the fusional area, an area that isincluded in a range defined by the near side fusion limit and the farside fusion limit in the depth direction.

The segmentation unit 110 segments the three-dimensional image in thedepth direction, using the output of the depth calculation unit 104 andthe fusional area determined by the fusional area determination unit109. In the present embodiment, the segmentation unit 110 segments thethree-dimensional image into three areas which are the fusional area, anarea on the nearer side than the fusional area, and an area on thefarther side than the fusional area.

The correction process determination unit 111 determines an imageprocessing method for each of the areas obtained by the segmentationunit 110 segmenting the three-dimensional image. Specifically, thecorrection process determination unit 111, for example, determines animage processing method to suppress display of an object as the imageprocessing method for the areas on the nearer and farther sides than thefusional area.

The image processing unit 112 performs image processing for each area,in accordance with the image processing method determined by thecorrection process determination unit 111.

As such, the correction process determination unit 111 and the imageprocessing unit 112 perform the processing of correcting thethree-dimensional image so as to suppress the display of the object thatis included in the three-dimensional image outside the fusional area. Inother words, the correction process determination unit 111 and the imageprocessing unit 112 correspond to the correction unit.

The display unit 113 displays the three-dimensional image processed bythe image processing unit 112. Examples of the display unit 113 includea three-dimensional display without dedicated glasses (the naked eye) orwith dedicated glasses, and a head mounted display. The display unit 113presents a left-eye image to the left eye and presents a right-eye imageto the right eye, thereby achieving three-dimensional display.

<Operation>

Next, the processing operation of the three-dimensional display device100 configured as set forth above will be described.

FIG. 13 is a flowchart illustrating processing operation of thethree-dimensional display device 100 according to the embodiment 2. Thethree-dimensional display device 100, for example, starts the followingprocessing, in accordance with a display start indication which isreceived by display start input means (not shown) from the viewer.

The viewpoint input unit 102 receives input of viewer's viewpointoperation (S110). Step S110 is repeated until input of viewer'sviewpoint operation is received. If input of viewpoint operation isreceived in step S110, the gaze point detection unit 106 detectsthree-dimensional image coordinates of the point of gaze of the viewerin the three-dimensional image, based on the viewer's eye movementobtained by the sensor 105 (S120). A method to detect the point of gazewill be described below.

The image transformation center determination unit 107 determines thethree-dimensional image coordinates of the point of gaze detected instep S120 as three-dimensional image coordinates of the central pointfor viewpoint change (S130). The central point for viewpoint change is apoint a display position at which is unchanged in changing a viewpointof the three-dimensional image. The three-dimensional image coordinatesare transformed into normal coordinates for defining a three-dimensionalmodel in the image information 101 a.

The viewpoint changing unit 103 extracts the image information 101 a onthe currently displayed three-dimensional image from the imageinformation storage unit 101. Then, the viewpoint changing unit 103changes the viewpoint of the displayed three-dimensional image, usingthe central point for viewpoint change determined in step S130 and aviewpoint specified by the input of the viewpoint operation received instep S110 (S140).

The depth calculation unit 104 calculates the depth of an object, whichis included in the three-dimensional image undergone the viewpointchange performed in step S140, in the three-dimensional coordinatesystem shown in FIG. 11 (S150).

The fusional area determination unit 109 refers to the fusional areainformation 108 a to determine a fusional area corresponding to theposition of the point of gaze in the depth direction detected in stepS120. Then, in accordance with the depth calculated in step S150, thesegmentation unit 110 segments the object included in thethree-dimensional image into three areas which are an area on the sidefarther away from the viewer than the fusional area is, the fusionalarea, and an area on the side closer to the viewer than the fusionalarea is (S160).

The correction process determination unit 111 allocates correctionprocesses to the respective three areas obtained by segmenting theobject in step S160 (S170).

The image processing unit 112 performs, for each area determined in stepS170, the correction process on the three-dimensional image undergonethe viewpoint change performed in step S140 (S180). Then, the imageprocessing unit 112 outputs the corrected three-dimensional image to thedisplay unit 113.

The display unit 113 displays the three-dimensional image output fromthe image processing unit 112 (S190).

As such, the viewpoint changing unit 103 changes the viewpoint of thethree-dimensional image about the point of gaze of the viewer as thecentral point, thereby changing the view point of the three-dimensionalimage so that the object which the viewer is gazing does not move. Thus,the viewpoint changing unit 103 can prevent the viewer from losing, dueto the viewpoint change, the object which the viewer is gazing, reducingload imposed on the viewer.

In a conventional method for changing a viewpoint of a three-dimensionalimage, image information is fixed on a three-dimensional coordinateaxis, and the three-dimensional image is rotated or zoomed about a fixedcentral point (e.g., origin of a fixed coordinate system). This iseffective in displaying the three-dimensional image, mapping it into twodimensions, because coordinates in the display screen and coordinates ina space where the viewer exists are independent of each other, theviewer views the display screen as if the viewer looks into a box, andchanging a viewpoint is identical to rotating the box and moving the boxtoward and away from the viewer. On the other hand, in three-dimensionaldisplay of a stereo image, a three-dimensional axis of the stereo imageis the same as a coordinate axis in the space where the viewer exists.Thus, changing the viewpoint of the three-dimensional image brings astate as if a view around the viewer is transformed. Inherent desire ofa viewer in changing a viewpoint is moving a viewer's position relativeto an object and changing positions of the viewer's eyes to view theobject from a different angle. When the viewer walks around and looksinto an object, the gaze object in a field of view of the viewer doesnot move at the center of the field of view while a coordinate axiswhich forms the field of view of the viewer rotates or extends andcontracts. In three-dimensional image display by stereoscopy, it isnecessary to transform the coordinate axis about the point of gaze as ifthe viewer walks around and looks into the point of gaze, withoutchanging the positions of the viewer's eyes. Thus, as in the presentembodiment, the viewpoint of the three-dimensional image is changedabout the point of gaze of the viewer as the central point, therebyappropriately changing the viewpoint of the three-dimensional image.

<Detect Point of Gaze>

Hereinafter, a method to detect the point of gaze of the viewer in stepS120 will be described in detail.

Information on eye movement depends on a type of the sensor 105. Themethod to detect the point of gaze depends on information on eyemovement. Here, as examples of detection of a point of gaze, methods todetect a point of gaze will be described in respective cases where: (a)the sensor 105 is a stationary camera installed in a housing of adisplay which includes a screen; (b) the sensor 105 is a proximitycamera which is worn by the viewer over the head; (c) the sensor 105 iselectrooculogram measuring means which measures eye movement fromcorneo-retinal potential differences of the eyes by electrodes incontact with a skin; and (d) the sensor 105 is a contact lens with acoil attached thereto.

If the sensor 105 is a stationary camera attached to a display asillustrated in FIG. 8A, the camera captures an image in front of thedisplay at the center. The gaze point detection unit 106 performs facedetection on the captured image, thereby extracting a face image of aviewer viewing the display. The gaze point detection unit 106 furtherextracts images of the viewer's eyes from the extracted face image.Then, the gaze point detection unit 106 identifies positions of pupilsor irises in the images of the viewer's eyes and calculates a pupillarydistance. The gaze point detection unit 106 obtains a position of apoint of gaze on a plane horizontal to the screen from the orientationof the face and the orientations of the midpoints of the eyes at theface detection, and obtains a position of the point of gaze in the depthdirection from the pupillary distance.

If the sensor 105 is a camera attached to each of glasses or goggles asillustrated in FIG. 8B, the left and right cameras capture images of theleft and right eyes, respectively. The gaze point detection unit 106obtains orientations of the left and right eyes from the captured leftand right images, respectively. The gaze point detection unit 106obtains, as the point of gaze, a point of intersection of linear linesindicating the respective orientations of the eyes. Alternatively, thegaze point detection unit 106 may derive positions of the pupils of theleft and right eyes, derive a position of the point of gaze in the planehorizontal to the screen from components of upward, downward, leftward,and rightward offsets of the pupil positions which are common to theeyes, and obtain a position of the point of gaze in the depth directionfrom an interocular distance.

If the sensor 105 is an electrooculogram measuring means as illustratedin FIG. 9, electrodes for measuring electrooculograms are attached incontact with both sides of each eye and a forehead above and a cheekbelow at least one eye. Then, the electrooculogram measuring meansmeasures changes in potential associated with eye movements in thehorizontal direction from two electrodes attached on the both sides ofeach eye. Then, the electrooculogram measuring means measures changes inpotential associated with eye movement in the vertical direction fromthe two electrodes attached on skins above and below the eye. The gazepoint detection unit 106 derives a position of the point of gaze in theplane horizontal to the screen from components of changes in potentialwhich are common to the eyes, and derives a position of the point ofgaze in the depth direction from components of changes in potentialwhich are in antagonism in the horizontal direction between the eyes.

If the sensor 105 is a contact lens with a coil attached thereto asillustrated in FIG. 10, the sensor 105 measures, for each eye, changesin distribution of potential associated with eye movement from the coil.The gaze point detection unit 106 derives a position of the point ofgaze in the plane horizontal to the screen from components of changes inpotential which are common to both eyes, and derives a position of thepoint of gaze in the depth direction from components of changes inpotential which are in antagonism in the horizontal direction betweenthe eyes.

It should be noted that the method to detect a point of gaze is notlimited to the above methods. For example, the gaze point detection unit106 may detect a point of gaze, using, as the sensor 105, myopotentialmeasuring means which measures movements of muscles around the eyes fromelectrodes in contact with a skin. Moreover, the gaze point detectionunit 106 may detect a point of gaze, based on potential associated withpupil motility induced by a contact lens embedded with a magnetic coil.Moreover, the gaze point detection unit 106 may detect a point of gazefrom potential associated with accommodation of crystalline lens inducedby a magnetic coil. Moreover, the gaze point detection unit 106 maydetect a point of gaze, using some combinations of the plurality ofdetection methods described above.

<Viewpoint Change>

FIG. 14 is a flowchart illustrating processing operation of theviewpoint changing unit 103 according to the embodiment 2. In otherwords, FIG. 14 illustrates details of step S140. Hereinafter, an exampleof a method to change the viewpoint of the three-dimensional image instep S140 will be described in detail.

The three-dimensional image is, as shown in FIG. 7, generated using theimage information 101 a in which coordinates indicating a position of apoint in a standard coordinate system representing a three-dimensionalspace and a display color of the point are paired. The image information101 a includes at least information on coordinates indicating positionsof points on a surface of the object.

For example, polygon rendering is used in the image generation process.In polygon rendering, three-dimensional shape data is represented by agroup of polygons. The three-dimensional image is generated byperforming a rendering process using the image information 101 a asshown in FIG. 7 represented by vertex coordinates of polygons. Inpolygon rendering, an object is represented by triangular planes eachformed of three points. For example, as shown in (a) of FIG. 15, anobject is represented by placing triangles in the standard coordinatesystem. A processing unit includes the plane and vertex coordinates ofeach triangle. Polygon rendering is a general one as a three-dimensionalgraphics method.

The viewpoint changing unit 103, first, determines a position at themiddlepoint of a line segment extending between the viewer's left andright eyes in the standard coordinate system, based on the viewpointoperation input in step S110 and the central point for viewpoint changedetermined in step S130 (S141). The viewpoint changing unit 103 sets themiddlepoint between the eyes to the viewpoint, as shown in (a) of FIG.15. Next, the viewpoint changing unit 103 determines positions of botheyes. The viewpoint changing unit 103 sets a distance between theviewer's left and right eyes to 6.5 cm, for example. The viewpointchanging unit 103 determines the positions of both eyes on a linear lineparallel with the horizontal axis (i.e., the X axis of FIG. 11) of thedisplay image so that the position determined in step S141 is in themiddle of the eyes. The viewpoint changing unit 103 rotates the axes ofthe normal coordinates to align with the orientations of the axes of thethree-dimensional display coordinates as shown in (b) of FIG. 15.

The viewpoint changing unit 103 determines orientations from thepositions of both eyes to the central point for viewpoint changedetermined in step S130, and viewing angles of both eyes, and determinesa display size of the object (S142). If the viewpoint operation input instep S110 includes zoom operation, the viewpoint changing unit 103resizes the display size of the object at which time the viewpointchanging unit 103 does not change the positions of the eyes of theviewer viewing the image.

Furthermore, the viewpoint changing unit 103 transforms the coordinatesso that a line extending between the position at the middlepoint betweenthe viewer's left and right eyes determined in step S141 and the centralpoint for viewpoint change determined in step S130 is parallel with theZ axis of FIG. 11 (S143). The viewpoint changing unit 103 performs thecoordinate transformation, assuming that the viewer views the image infront of the screen.

In the coordinates obtained by the transformation in step S143, theviewpoint changing unit 103 determines a range (size) of the object thatcan be projected onto the screen in accordance with the positions andorientations of the eyes (S144).

<Determine Correction Process>

FIG. 16 is a flowchart illustrating processing operation of thecorrection process determination unit 111 according to the embodiment 2.In other words, FIG. 16 illustrates details of step S170.

The correction process determination unit 111 determines a correctionprocess to be performed on each of three areas obtained as a result ofthe segmentation in step S160. In other words, the correction processdetermination unit 111 determines a correction process to be performedon each point on the surface of the object included in thethree-dimensional image undergone the viewpoint change performed in stepS140. It should be noted that in the present embodiment, thesegmentation is performed by comparing a Z-coordinate value in thecoordinate system of FIG. 11 with the fusional area.

First, the correction process determination unit 111 determines whetherthe point on the surface of the object included in the three-dimensionalimage is included in an area which has a depth value smaller than thenear side fusion limit (S171). In other words, the correction processdetermination unit 111 determines whether a Z-coordinate value of thepoint in the coordinate system of FIG. 11 which indicates a position ona side closer to the viewer's eyes than a position, indicated by aZ-coordinate value of the near side fusion limit, is.

If the Z-coordinate value of the point indicates a position on the sidecloser to the viewer's eyes than the position indicated by theZ-coordinate value of the near side fusion limit is (Yes in S171), thecorrection process determination unit 111 determines that a correctionprocess to be performed on that point is hiding the point (S172). Thehiding refers to removing the point from the display range. In thiscase, a surface of the object where that point is included is handled asif transparentized.

On the other hand, if the Z-coordinate value of the point is equal tothe Z-coordinate value of the near side fusion limit or indicates aposition on the side farther away from the viewer's eyes (No in S171),the correction process determination unit 111 determines whether theZ-coordinate value of the point indicates a position on the side fartheraway from the viewer's eyes than a position, indicated by theZ-coordinate value of the far side fusion limit, is (S174).

If the Z-coordinate value of the point indicates a position on the sidefarther away from the viewer's eyes than the position, indicated by theZ-coordinate value of the far side fusion limit, is (Yes in S174), thecorrection process determination unit 111 determines that a correctionprocess to be performed on that point is blurring (S175). On the otherhand, if the Z-coordinate value of the point is equal to theZ-coordinate value of the far side fusion limit or indicates a positionon the side closer to the viewer's eyes than the position, indicated bythe Z-coordinate value of the far side fusion limit, is (No in S174),the correction process determination unit 111 determines that thecorrection process is not to be performed on that point (S176).

It should be noted that specifically, for example, the blurring is aprocess of low pass filtering a target area to reduce a spatialfrequency. The lower the low pass frequency is the greater an extent towhich the target area is blurred. Another example of the blurring is aprocess of mixing and blurring different images A and B, which arerespectively a right-eye image and a left-eye image, and displayresultant images. As blurring, for example, a process can be used inwhich a value of color information at each coordinate point of thetarget area is blurred using an intermediate color of color informationof the image A and color information of the image B. A degree of theblurring is determined based on a ratio of mixing the color of one ofthe images to the other. The degree of the blurring is largest when themixing ratio is 1:1.

<Image Processing>

FIG. 17 is a flowchart illustrating processing operation of the imageprocessing unit 112 according to the embodiment 2. In other words, FIG.17 illustrates details of step S180.

The image processing unit 112 hides an object located in an area on anearer side than a plane which is in parallel with the display screenand includes the near side fusion limit (S181). In polygon rendering,points within that area are removed from a range for image generation,thereby hiding the object. Moreover, the image processing unit 112 cangenerate an image of the blood vessel's cross-section and inside theblood vessel if the image information 101 a includes data of pointsinside the blood vessel.

Next, the image processing unit 112 performs polygon rendering on thethree-dimensional image (the three-dimensional model) undergone theviewpoint change performed in step S140, thereby generating a right-eyeimage and a left-eye image. In other words, the image processing unit112 applies lighting to the three-dimensional model, in which the objectincluded in the area on the nearer side than the fusional area ishidden, by a light source which has an attribute and a fixed position ona display coordinate axis (S182).

Using light of a solid-state light source and a display color for eachof coordinate points included in the image information 101 a, the imageprocessing unit 112 determines a color of each coordinate point.Subsequently, the image processing unit 112 generates triangles whichform a polygon, and fill the triangles with pixels. The fillinginterpolates a color for each pixel based on colors of vertices (S183).The image processing unit 112 applies blurring, by low pass filtering,to each of the pixels generated by the points included in the area onthe farther side than the fusional area (i.e., the area on the sidefarther away from the viewer) (S184).

An object located on the farther side than the fusional area also causesdouble vision. However, an object located on a farther side than thepoint of gaze does not hide the object located at the point of gaze.Thus, while double vision does not significantly impede the viewer fromseeing the object located at the point of gaze, the viewer feelsuncomfortable, experiencing double vision.

An increase in depth does not necessarily increase a disparity of anobject to infinity. The human eyes converge to see an object. However,in an opposite movement, which is looking into the distance(divergence), the left and right eyes does not move to angles at whichthe left and right eyes are in opposite directions. A state where thedivergence is at a maximum value indicates that viewing directions ofthe left and right eyes are parallel with each other. Thus, changes indisparity corresponding to changes in depth on a far side in thethree-dimensional image are small as compared to changes in disparitycorresponding to changes in depth on a near side. Thus, double vision onthe far side is not as large as double vision produced on the near side.

Thus, the image processing unit 112, in step S184, blurs an objectlocated on the farther side than the fusional area by, for example, amethod of lowering a spatial frequency by a filtering process to causethe object fell into a state acceptable as a background image.

The image processing unit 112 performs processing on the right-eye imageand the left-eye image in step S180 to generate a stereoscopic image.

FIG. 18 shows diagrams illustrating examples of the three-dimensionalimage which is displayed by the three-dimensional display device 100according to the embodiment 2. Specifically, (a) of FIG. 18 shows anuncorrected three-dimensional image, and (b) of FIG. 18 shows acorrected three-dimensional image.

As FIG. 18 shows, when a blood vessel is displayed as if extending fromthe front of the screen into the back as illustrated in (b) of FIG. 1,the three-dimensional display device 100 can prevent a near side portionof the blood vessel from looking double as illustrated in (a) of FIG.18. Furthermore, the three-dimensional display device 100 can display animage of a cross-section of the blood vessel cut along a plane of thenear side fusion limit as illustrated in (b) of FIG. 18. This allows thethree-dimensional display device 100 to display a junction between theremoved near side portion of the blood vessel and a portion within thefusional area so that the junction appears natural.

<Effects>

As described above, according to the three-dimensional display device100 of the present embodiment, the viewpoint changing unit 103 changesthe view point of the three-dimensional image using the point of gaze ofthe viewer as the central point for the viewpoint change, therebyachieving viewpoint movement about the point of gaze. Furthermore, thethree-dimensional display device 100 can determine a fusional area inaccordance with a position of the point of gaze, remove an objectlocated on the nearer side than the fusional area, and blur an objectlocated on the farther side than the fusional area. This can suppressthe occurrence of double vision produced by the objects located outsidethe fusional area. Furthermore, the three-dimensional display device 100can improve the visibility of an object located at the point of gaze byremoving an object on the nearer side that is hiding the object locatedat the point of gaze. By blurring an object located on the farther sidethan the fusional area, the three-dimensional display device 100 canalso correct the three-dimensional image so that the presence of theblurred object is known as a blur background image. This allows thethree-dimensional display device 100 to avoid an image after theviewpoint change from causing discomfort to the viewer as if the imagehas the same depth perception as that produced by an image before theviewpoint change. From the above, the three-dimensional display device100 can change the viewpoint of the three-dimensional image so that theobject located at the point of gaze does not move, ensuring thevisibility of the object located at the point of gaze and retaining thenaturalness throughout the image.

In the present embodiment, the fusional area information 108 a ispre-stored in the fusional area information storage unit 108. An areawhere the fusion occurs without moving the eyes may vary from individualto individual. Moreover, it is necessary that a viewer can reliablyascertain the context of the blood vessel described in the presentembodiment when viewing the image of the blood vessel. Thus, a narrowerfusional area is more suitable. Moreover, a fusional area of the sameindividual may vary over time due to fatigue and so on. Thus, thefusional area determination unit 109 may store time-series data of thefusional area when the viewer is viewing a three-dimensional video intothe fusional area information storage unit 108, and determine a fusionalarea narrower than the stored fusional areas. In particular, correctionof a three-dimensional video using a narrower fusional area is effectivefor use in a long surgery or the like in operation.

While in the present embodiment, the three-dimensional image includes ablood vessel object representing a blood vessel, it should be noted thatthe three-dimensional image may not include a blood vessel object. Forexample, the three-dimensional image may include objects of tissues ofthe abdomen, the thorax, or the head, for example. By including objectsof body regions different from blood vessels in the three-dimensionalimage as such, the three-dimensional display device 100 is applicable tosurgeon simulators for surgical preparation or educational purpose.

A surgeon simulator, as with PTL 3 (International PublicationWO2010/021309), for example, stores three-dimensional image data ofpatient's body in association with elasticity data and so on, anddeforms and displays a polygon model image in accordance with surgicalprocedure such as incision. In the surgeon simulator also, when, forexample, a viewer is viewing a simulation image of a patient's abdomen,as the viewer changes the viewpoint from a side of the patient towardthe head, the thorax and the head are located on nearer sides to theviewer than abdominal organs gazed by the viewer. In such a case, bodyparts located on the near sides look double, not only hiding an organ atthe point of gaze but also making the viewer feel discomfort by flickeror the like.

Thus, application of the three-dimensional display device 100 accordingto the present embodiment to the surgeon simulator suppresses display ofobjects located outside a fusional area, thereby displaying athree-dimensional image without causing discomfort to the viewer, whileretaining information on arrangement of organs.

While in the present embodiment, the operation of the three-dimensionaldisplay device 100 has been described in terms of rotation of an imageof a blood vessel by way of example of the viewpoint change, it shouldbe noted that the viewpoint change may be a process other than rotation,such as a zoom operation. Operation of the three-dimensional displaydevice 100 in a zoom process on an image of a blood vessel is the sameas that at rotation of an image. If the zoom process is input in stepS110, a point of gaze of a viewer is detected in step S120 and set to acentral point for the zoom process in step S130. The zoom process isperformed in step S140 by the viewer moving a position at themiddlepoint between the eyes closer to or away from the point of gaze,and thereby the viewpoint of the three-dimensional image is changed.

FIG. 19 shows schematic views for illustrating viewpoint changeaccording to a variation of the embodiment 2. Specifically, FIG. 19 is aschematic view of the zoom process. Part (a) of FIG. 19 illustrates arelationship between the zoom process (enlarging or reducing) and adirection of movement of the viewpoint in the standard coordinatesystem. In the standard coordinate system, the closer the coordinateposition of the viewpoint is to a coordinate position of an object, thegreater the image of the displayed object is enlarged. The farther thecoordinate position of the viewpoint is away from the coordinateposition of the object in the standard coordinate system, the smallerthe size of the image of the displayed image is reduced.

Part (b) of FIG. 19 is a schematic view of changes in display in thethree-dimensional display coordinate system at the enlargement process.The dotted cylinder is an object before the viewpoint change, and thewhite cylinder is an object after the enlargement process. Theenlargement process moves the near side of the cylinder closer to aviewer's viewing position and moves the far side of the cylinder fartheraway from the viewing position.

Part (c) of FIG. 19 is a schematic view of changes in display in thethree-dimensional display coordinate system at the reduction process.Similarly to (b) of FIG. 19, the dotted cylinder is an object prior tothe viewpoint change, and the white cylinder is an object after thereduction process. The reduction process moves the near side of thecylinder farther away from a viewer's viewing position and moves the farside of the cylinder closer to the viewing position.

As the example of FIG. 19 shows, the zoom process also changes a depthrange of an object. Thus, effects similar to that obtained from therotation process can be achieved by performing, similarly to therotation process, image processing on a three-dimensional image based ona fusional area.

Embodiment 3

In the present embodiment, a three-dimensional image includes aplurality of blood vessel objects representing a plurality of bloodvessels. A three-dimensional display device corrects thethree-dimensional image, based on the importance levels of blood vesselobjects.

FIG. 20 is a block diagram of a functional configuration of athree-dimensional display device 200 according to an embodiment 3.

The three-dimensional display device 200 includes an image informationstorage unit 201, a viewpoint input unit 102, a viewpoint changing unit103, a depth calculation unit 104, a sensor 105, a gaze point detectionunit 106, an image transformation center determination unit 107, afusional area information storage unit 108, a fusional areadetermination unit 109, a blood vessel connection information extractionunit 202, a blood vessel importance calculation unit 203, a segmentationunit 204, a correction process determination unit 205, an imageprocessing unit 206, and a display unit 113.

The image information storage unit 201 is, for example, a hard diskdrive or a semiconductor memory. The image information storage unit 201stores image information 201 a on a three-dimensional image which isdisplayed.

The image information 201 a includes information representingthree-dimensional models of the blood vessel objects and connectioninformation indicating connectivity between the blood vessel objects.Specifically, the image information 201 a includes coordinates of pointson each blood vessel object, a display color, a blood vessel ID, and abranch number, for example, as shown in FIG. 21.

The blood vessel connection information extraction unit 202 is by way ofexample of a blood vessel connection information obtaining unit. Theblood vessel connection information extraction unit 202 refers to theimage information 201 a to obtain connection information indicative ofconnectivity of a blood vessel object located at a point of gaze to eachof the blood vessel objects included in the three-dimensional image.Specifically, the blood vessel connection information extraction unit202 extracts, from the image information 201 a, connection informationof the blood vessel object which the viewer is gazing with other bloodvessel objects, based on coordinate information of the point of gaze ofthe viewer detected by the gaze point detection unit 106 and positionsof the blood vessel objects on a currently displayed three-dimensionalimage.

The blood vessel importance calculation unit 203 calculates theimportance of each of the blood vessel objects in the three-dimensionalimage, based on a fusional area determined by the fusional areadetermination unit 109, the connection information extracted by theblood vessel connection information extraction unit 202, and depthinformation of the blood vessel object.

Specifically, the blood vessel importance calculation unit 203calculates, for each of the blood vessel objects, the importance of theblood vessel object so that the blood vessel object, if included in thefusional area, is of higher importance than if the blood vessel objectis not included in the fusional area.

Moreover, the blood vessel importance calculation unit 203 maycalculate, for each of the blood vessel objects, the importance of theblood vessel object so that, for example, the blood vessel object havinga less number of blood vessel branches to the blood vessel objectlocated at the point of gaze is of higher importance, wherein the numberof blood vessel branches is the number of branches and connections of ablood vessel which appear when connecting two blood vessel objects in ashortest path.

Moreover, the blood vessel importance calculation unit 203 maycalculate, for each of the blood vessel objects, the importance of theblood vessel object so that, for example, the blood vessel object havinga smaller spatial distance to the blood vessel object located at thepoint of gaze is of higher importance.

The segmentation unit 204 segments the three-dimensional image intoareas in a depth direction, in accordance with the importance levels ofthe blood vessel objects calculated by the blood vessel importancecalculation unit 203.

The correction process determination unit 205 determines how to processthe three-dimensional image for each of the areas obtained by thesegmentation by the segmentation unit 204.

In other words, the correction process determination unit 205 and theimage processing unit 112 perform processing to correct thethree-dimensional image so that display of a blood vessel object theimportance of which is lower is suppressed to a greater extent.

The image information 201 a shown in FIG. 21 includes coordinates ofpoints on each blood vessel object, a display color, a blood vessel ID,and a branch number. The coordinates are expressed in the standardcoordinate system. The display color indicates a display color of acorresponding point. The blood vessel ID is a sign or a number uniquelyattached to a blood vessel object in the three-dimensional image.

The branch number is set according to a predetermined rule and indicatesconnectivity between blood vessels. In other words, the branch numbercorresponds to the connection information. For example, as shown in FIG.22, the branch number “0” is attached to the origin of the blood vesselin the image and single-digit numbers in sequence are attached to bloodvessels sequentially branching off from the blood vessel that have thebranch number “0.” Furthermore, the number of digits to be attached to abranch number is increased each time the blood vessel branches. Thenumber of digits of the branch number attached as such indicates thenumber of times the blood vessel branches off starting from the origin.In other words, the branch number indicates connectivity between theblood vessels.

It should be noted that the expression of the branch information is notlimited to the above. The expression of branch information (theconnection information) may be other than the above expression, insofaras a blood vessel object from which each blood vessel object hasbranched off can be determined.

FIG. 23 is a flowchart illustrating processing operation of thethree-dimensional display device 200 according to the embodiment 3.

After processing of steps S110 to S150 is performed, the blood vesselconnection information extraction unit 202 identifies a blood vesselobject which the viewer is gazing, using coordinates indicating aposition of the point of gaze of the viewer detected in step S120, thecurrently displayed three-dimensional image, and normal coordinates inthe image information 201 a (S210). The blood vessel connectioninformation extraction unit 202 further refers to the image information201 a to extract connection information indicating connectivity betweenthe blood vessel object which the viewer is gazing and each of theplurality of blood vessel objects included in the three-dimensionalimage (S220).

The blood vessel importance calculation unit 203 calculates theimportance of each of the blood vessel objects included in thethree-dimensional image, based on the connection information extractedin step S220 and the fusional area (S230). The segmentation unit 204segments the three-dimensional image into a plurality of areas, based onthe importance levels calculated in step S230 (S260). The segmentationunit 204 attaches importance levels to the areas and outputs theimportance levels.

The correction process determination unit 205 determines a correctionprocess for each area obtained by the segmentation in step S260 (S270).The correction process is determined according to the importance of thearea.

FIG. 24 is a block diagram of a detailed functional configuration of theblood vessel connection information extraction unit 202 according to theembodiment 3. The blood vessel connection information extraction unit202 includes a coordinate transforming unit 202 a, a coordinate queryingunit 202 b, and a connection information extraction unit 202 c.

In step S220, first, the coordinate transforming unit 202 a obtains thecurrently displayed three-dimensional image and coordinate informationof the three-dimensional image from the image processing unit 206.Furthermore, the coordinate transforming unit 202 a obtains the imageinformation 201 a corresponding to the currently displayedthree-dimensional image from the image information storage unit 201.Furthermore, the coordinate transforming unit 202 a transforms normalcoordinates included in the image information 201 a into displaycoordinates of the currently displayed three-dimensional image. Next,the coordinate querying unit 202 b obtains the coordinates of the pointof gaze of the viewer from the gaze point detection unit 106. Then, thecoordinate querying unit 202 b compares the blood vessel object thenormal coordinates of which has been transformed into the displaycoordinates of the currently displayed three-dimensional image and thepoint of gaze of the viewer. The coordinate querying unit 202 bidentifies a blood vessel object located at a coordinate position thatis closest to the point of gaze, as the blood vessel object which theviewer is gazing. Furthermore, the coordinate querying unit 202 bextracts a blood vessel object within a predetermined proximity from thepoint of gaze as a blood vessel object proximate to the point of gaze.The connection information extraction unit 202 c extracts a blood vesselobject leading to the blood vessel object identified as the blood vesselobject which the viewer is gazing, based on the blood vessel ID and thebranch number that are included in the image information 201 a. Forexample, the connection information extraction unit 202 c extracts,among data items that have the same blood vessel ID as the blood vesselID of the blood vessel object which the viewer is gazing, a data itemhaving a branch number the leftmost digit of which is the same as thatof the blood vessel object which the viewer is gazing, therebyextracting a blood vessel object which has branched off, from the originof the blood vessel, at the same location as the blood vessel objectwhich the viewer is gazing.

FIG. 25 is a block diagram of a detailed functional configuration of theblood vessel importance calculation unit 203 according to the embodiment3.

The blood vessel importance calculation unit 203 includes a fusiondetermination unit 203 a, a distance calculation unit 203 b, a bloodvessel branch distance calculation unit 203 c, a score translation table203 d, and an adder 203 e.

FIG. 26 is a flowchart illustrating processing operation of the bloodvessel importance calculation unit 203 according to the embodiment 3.

First, using the depth (i.e., a Z coordinate value) of each point on theobject in the display coordinate system calculated in step S150, a depthposition of the near side fusion limit, and a depth position of the farside fusion limit obtained in step S120, the fusion determination unit203 a determines if the point on the blood vessel object is located onthe nearer side than the fusional area, within the fusional area, or onthe farther side than the fusional area (S231).

The distance calculation unit 203 b calculates, for each point on theblood vessel object extracted in step S220, an Euclidean distancebetween that point and the point of gaze (S232).

The blood vessel branch distance calculation unit 203 c calculates abranch distance of each blood vessel object extracted in step S220 tothe blood vessel object which the viewer is gazing identified in stepS210 (S233). The branch distance indicates the number of branches. Thebranch distance is calculated from a branch number, for example.Specifically, the blood vessel branch distance calculation unit 203 ccompares a branch number of each blood vessel object and the branchnumber of the blood vessel object which the viewer is gazing, andcalculates a branch distance therebetween by summing up values which areobtained by multiplying the leftmost digit of the branch number by10,000, the second leftmost digit by 1,000, the third leftmost digit by100, and the fourth leftmost digit by 10. If the number of digits of oneof branch numbers to be compared therebetween is less than the other,the blood vessel branch distance calculation unit 203 c calculates abranch distance, assuming that the lacking digit to be compared is zero.

It should be noted that the method to calculate a branch distance is notlimited to the above method as it can quantitatively determine how farto branch back the blood vessels from two points on blood vessel objectsto reach a same blood vessel.

The adder 203 e refers to the score translation table 203 d to obtainscores respectively corresponding to the result of the determination instep S231, the spatial distance calculated in step S232, and the branchdistance calculated in step S233 (S234). The score translation table 203d includes scores (incremented or decremented) corresponding to factorswhich determines the importance of a blood vessel, for example, as shownin FIGS. 27A, 27B, and 27C.

The adder 203 e, for example, refers to a first score translation tableshown in FIG. 27A to obtain a score (−1500, 0, or −600) corresponding tothe result of the determination in step S231. Likewise, the adder 203 erefers to, for example, a second score translation table shown in FIG.27B to obtain a score corresponding to the Euclidean distance calculatedin step S232. The adder 203 e also refers to, for example, a third scoretranslation table shown in FIG. 27C to obtain a score corresponding tothe branch distance calculated in step S233.

Scores included in each score translation table are predetermined by,for example, statistical learning or the like.

In the first score translation table, scores are set so that a bloodvessel object, if included in the fusional area, is of higher importancethan if the blood vessel object is not included in the fusional area.Moreover, in the first score translation table, scores are set so that ablood vessel object, if located on the farther side than the fusionalarea, is of higher importance than if the blood vessel object is locatedon the nearer side than the fusional area.

In the second score translation table, scores are set so that a bloodvessel object having a smaller spatial distance to the blood vesselobject located at the point of gaze is of higher importance.

In the third score translation table, scores are set so that a bloodvessel object having a less number of branches (the branch distance) tothe blood vessel object located at the point of gaze is of higherimportance.

The adder 203 e further calculates the importance of the blood vesselobject by adding the scores obtained in step S234 for each point on theblood vessel object (S235).

While in the present embodiment, the adder 203 e refers to the scoretranslation table to obtain scores, it should be noted that the adder203 e may obtain the scores using a predetermined transformationfunction.

FIG. 28 is a block diagram of a detailed functional configuration of thesegmentation unit 204 according to the embodiment 3.

The segmentation unit 204 includes a segmentation table 204 a and asegmentation process unit 204 b.

In step S260, first, the segmentation process unit 204 b refers to thesegmentation table 204 a to segment the three-dimensional image into aplurality of regions, in accordance with the importance levels of theblood vessel objects calculated in step S230.

For example, the segmentation process unit 204 b refers to thesegmentation table 204 a shown in FIG. 29 to segment the area on thenearer side than the fusional area into two regions and segment the areaon the farther side than the fusional area into three regions. It shouldbe noted that the segmentation table 204 a shown in FIG. 29 is anexample. The segmentation unit 204 may include the segmentation table204 a for segmenting the three-dimensional image into more areas.Moreover, the segmentation unit 204 may segment the fusional area into aplurality of regions. Moreover, while in the present embodiment, aboundary value of importance for the segmentation is a fixed value, theboundary value may be adaptively changed in accordance with adistribution of importance levels or a distribution of depth values inthe three-dimensional image.

While in the present embodiment, the area is segmented into a fixednumber of regions, it should be noted that the number of areas intowhich the area is segmented may be changed in accordance with a range ofthe distribution of depths of the three-dimensional image, a size of thedisplay range, or a size of the display screen.

In step S270, the correction process determination unit 205 refers tothe segmentation table 204 a to determine a correction process for eachof the areas obtained by the segmentation in step S260.

In step S180, the image processing unit 206 generates a left-eye imageand a right-eye image, in accordance with the correction process foreach area determined in step S270. For an area the determined correctionprocess on which is semi-transparentizing, the display color is madesemitransparent when generating vertex coordinates of an object. On theother hand, for an area the determined correction process on which isone of blurring 1, 2, and 3, pixels are generated and then blurring byfiltering is applied. The blurring 1 to 3 have different limitedfrequencies of a low pass filter. In the present embodiment, a limitedfrequency of the blurring 1 is the lowest, and a limited frequency ofthe blurring 3 is the highest.

Changing the correction process in accordance with the importance of ablood vessel as such can prevent fully hiding a blood vessel object thatis, although located on the nearer side than the fusional area, usefulto the viewer. Moreover, among the blood vessel objects located onfarther sides than the fusional area, a blood vessel object that isuseful to the viewer can be blurred by a reduced amount. Thus, thethree-dimensional display device 200 determines a correction process inaccordance with not only a depth but also the importance of a bloodvessel object, thereby improving the visibility of a three-dimensionalimage, while avoiding the loss of information on an important bloodvessel object which leads to a blood vessel object that is located at apoint of gaze.

<Effects>

As described above, according to the three-dimensional display device200 of the present embodiment, the visibility of a blood vessel objectthat not only has a small spatial distance to a blood vessel objectwhich the viewer is gazing but also has high connectivity to the bloodvessel object which the viewer is gazing can be improved. Thus, thevisibility of a blood vessel object that is useful to the viewer can beimproved. For example, the three-dimensional display device 200 canavoid the loss of information on a blood vessel object that has,although located outside the fusional area, a large connection with ablood vessel object which the use is gazing.

While in the present embodiment, the three-dimensional image issegmented in step S260 in accordance with the importance levelscalculated in step S230, it should be noted that the three-dimensionalimage may not be segmented. In this case, the three-dimensional imagemay be corrected by transforming the importance levels into imageprocessing parameters and performing image processing.

Embodiment 4

An embodiment 4 is different from the embodiment 3 in that theimportance of a blood vessel is determined taking a state of a medicalinstrument, such as a catheter, into account.

FIG. 30 is a block diagram of a functional configuration of athree-dimensional display device 300 according to the embodiment 4.

The three-dimensional display device 300 includes an image informationstorage unit 303, a viewpoint input unit 102, a viewpoint changing unit103, a depth calculation unit 104, a sensor 105, a gaze point detectionunit 106, an image transformation center determination unit 107, afusional area information storage unit 108, a fusional areadetermination unit 109, a blood vessel connection information extractionunit 202, a blood vessel importance calculation unit 305, a segmentationunit 204, a correction process determination unit 205, an imageprocessing unit 206, a display unit 113, an instrument informationstorage unit 302, an instrument traveling direction determination unit304, and an image and instrument information input unit 301.

The image and instrument information input unit 301 receives images of ablood vessel and a medical instrument, such as a catheter, when themedical instrument is being inserted into the blood vessel, andinformation (instrument information) on the medical instrument, andoutputs them to the instrument information storage unit 302 and theimage information storage unit 303. The images are input from an imagingdevice (not shown) such as a camera or X-ray imaging machine or arecording device (not shown). The instrument information is input from acontrol device (not shown) of the instrument, or an image processingsystem (not shown) or the like which obtains instrument information fromthe image of the instrument.

The instrument information storage unit 302 stores instrumentinformation 302 a which indicates, in a time sequence, positions of theleading end of a medical instrument, such as a catheter, which isadvanced through a blood vessel. The image information storage unit 303stores image information 303 a. The image information 303 a includesinformation which represents three-dimensional models of the bloodvessel object and the instrument object in time series, and connectioninformation which indicates connectivity between blood vessel objects.

FIG. 32 is a diagram showing an example of the instrument information302 a in the embodiment 4. As FIG. 32 shows, the instrument information302 a includes time information, coordinates of points on the bloodvessel object and the instrument object in the standard coordinatesystem, a display color, and leading-end information which indicates aposition of the leading end of the instrument object.

FIG. 33 is a diagram showing an example of the image information 303 ain the embodiment 4. As FIG. 33 shows, the image information 303 aincludes information, such as time information, coordinates of points onthe blood vessel object and the instrument object in the standardcoordinate system, a display color, a blood vessel ID, and a branchnumber of the blood vessel.

Here, the coordinates of points included in the image information 303 aare the same as the coordinates of points included in the instrumentinformation 302 a. The image information 303 a is synchronized in timewith the instrument information 302 a. It should be noted that theinstrument information 302 a and the image information 303 a may not betime-series information and may be information at arbitrary time. Inother words, the three-dimensional image may be a still image ratherthan a video.

The instrument traveling direction determination unit 304 is by way ofexample of an identification unit. The instrument traveling directiondetermination unit 304 determines a relationship of the instrumentobject with each blood vessel object, based on the position of theleading end of the instrument object indicated by the instrumentinformation 302 a. Specifically, the instrument traveling directiondetermination unit 304 identifies, among a plurality of blood vesselobjects, a blood vessel object though which the instrument object hasalready passed, or a blood vessel object through which the instrumentobject does not pass.

FIG. 31 is a block diagram of a detailed functional configuration of theblood vessel importance calculation unit 305 according to the embodiment4.

The blood vessel importance calculation unit 305 includes the fusiondetermination unit 203 a, the distance calculation unit 203 b, the bloodvessel branch distance calculation unit 305 c, a score translation table305 d, and an adder 305 e.

The blood vessel importance calculation unit 305 calculates theimportance of a blood vessel object, based on a fusional area determinedby the fusional area determination unit 109, a depth of the blood vesselobject calculated by the depth calculation unit 104, connectioninformation extracted by the blood vessel connection informationextraction unit 202, and the information on the travel of the instrumentdetermined by the instrument traveling direction determination unit 304.

The three-dimensional display device 300 according to the presentembodiment is different from the embodiment 2 in the operation of theinstrument traveling direction determination unit 304, and the operationof the blood vessel importance calculation unit 305 using output of theinstrument traveling direction determination unit 304.

The instrument traveling direction determination unit 304, first,transforms coordinates of the instrument object as with the blood vesselobject. Then, the instrument traveling direction determination unit 304identifies a blood vessel object closest to a position of the leadingend of the instrument object, as a blood vessel object having theinstrument inserted therein.

For example, in FIG. 32, the leading end of the instrument object is atcoordinates (22, 18, 173) at time 00:12:54.06. FIG. 33 indicates that ablood vessel object having a blood vessel ID “A01” and a branch number“111” is displayed at coordinates (22, 18, 173) at time 00:12:54.06.

The instrument traveling direction determination unit 304 refers to theimage information 303 a and the instrument information 302 a to obtain ablood vessel ID and a branch number of the blood vessel object in whichthe leading end of the instrument object is present. The instrumenttraveling direction determination unit 304 refers to the instrumentinformation 302 a at a time backwards in time by a fixed time (e.g., 30seconds) from the time at which the current displayed three-dimensionalimage is captured to obtain coordinates of a position of the leading endof the instrument object at that time backwards in time by the fixedtime. The instrument traveling direction determination unit 304identifies a blood vessel object in which the presence of the leadingend of the instrument object at that time backwards in time by the fixedtime is known, from the coordinates of the position of the leading endof the instrument object at the time backwards in time by the fixed timeand the image information 303 a.

In FIG. 32, for example, the instrument traveling directiondetermination unit 304 refers to the instrument information 302 a attime 00:12:24.06 backwards in time by seconds from the time 00:12:54.06to identify coordinates (21, 19, 187) indicating a position of theleading end of the instrument object 30 seconds prior to the time00:12:54.06. Then, the instrument traveling direction determination unit304 refers to the image information 303 a of FIG. 33 to obtain a bloodvessel ID “A01” and a branch number “11” of the blood vessel object atthe coordinates (21, 19, 187) at the time 00:12:24.06. The instrumenttraveling direction determination unit 304 extracts a blood vesselobject (in the example of FIG. 33, the blood vessel object having theblood vessel ID “A01” and the branch number “11”) in which the presenceof the leading end of the instrument object is known in the past, and ablood vessel object (in the example of FIG. 33, the blood vessel objecthaving the blood vessel ID “A01” and the branch number “111”) in whichthe leading end of the instrument object is currently present, andidentifies a blood vessel ID and a branch number of a blood vesselobject through which the leading end of the instrument object hasalready passed between the two time points. In the example of FIG. 33,the blood vessel object having the blood vessel ID “A01” and the branchnumber “11” and the blood vessel object having the blood vessel ID “A01”and the branch number “111” are the blood vessel objects through whichthe leading end of the instrument object has already passed.

The instrument traveling direction determination unit 304 furtherdetermines a blood vessel object through which the instrument object islikely to pass in the future, and a blood vessel object that isunrelated to the passage of the instrument object, using the identifiedblood vessel ID and the identified branch number. The blood vesselobjects unrelated to the passage of the instrument object is a bloodvessel object through which the instrument object does not pass. Inother words, the blood vessel object unrelated to the passage of theinstrument object is a blood vessel object through which the instrumentobject is unlikely to pass.

A blood vessel ID and a branch number include information whereby ablood vessel object from which each blood vessel object has branched offcan be determined, for example, as shown in FIG. 22. In the example ofFIG. 22, the instrument object passes through the blood vessel objectthat has the blood vessel ID “A01” and the branch number “11” and theblood vessel object that has the blood vessel ID “A01” and the branchnumber “111.” Thus, it can be determined that the instrument object isadvancing from the origin of the blood vessel in a direction in whichthe branch number increases.

Thus, the instrument traveling direction determination unit 304determines blood vessel objects that have the blood vessel ID “A01,” andbranch numbers the first three digits of which are “111” as blood vesselobjects through which the instrument object is likely to pass in thefuture. The instrument traveling direction determination unit 304 alsodetermines the other blood vessel objects as the blood vessel objectunrelated to the passage of the instrument object. The instrumenttraveling direction determination unit 304 outputs to the blood vesselimportance calculation unit 305 information which indicates the bloodvessel object through which the leading end of the instrument object hasalready passed, the blood vessel object through which the leading end ofthe instrument object is likely to pass in the future, and the bloodvessel object unrelated to the passage of the leading end of theinstrument object.

The blood vessel importance calculation unit 305 obtains the fusionalarea from the fusional area determination unit 109, obtains connectioninformation from the blood vessel connection information extraction unit202, and obtains the depths of the blood vessel objects from the depthcalculation unit 104. Furthermore, the blood vessel importancecalculation unit 305 obtains, from the instrument traveling directiondetermination unit 304, the information indicating the blood vesselobject through which the leading end of the instrument object haspassed, the blood vessel object through which the instrument object islikely to pass, and the blood vessel object unrelated to the passage ofthe instrument object.

A blood vessel branch distance calculation unit 305 c, as with step S233in the embodiment 3, calculates a branch distance. The blood vesselbranch distance calculation unit 305 c further attaches, to the bloodvessel object having the branch distance calculated, the informationthat is output from the instrument traveling direction determinationunit 304, indicating the blood vessel object through which the leadingend of the instrument object has already passed, the blood vessel objectthrough which the instrument object is likely to pass in the future, andthe blood vessel object unrelated to the passage of the instrumentobject.

The adder 305 e refers to the score translation table 305 d to obtainscores, wherein the adder 305 e performs processing to decrease scoresof the blood vessel object through which the leading end of theinstrument object has already passed and the blood vessel objectunrelated to the passage of the instrument object, and increase scoresof the blood vessel object through which the instrument, object islikely to pass in the future.

FIGS. 34A to 34E are diagrams showing examples of the score translationtable 305 d in the embodiment 4. The score translation table 305 dincludes, for example, a first score translation table of fusional area,a second score translation table of spatial distance, and third, fourthand fifth score translation tables of branch distances. The third,fourth and fifth score translation tables are thus classified based onrelationships between the branch distances and the passage of theinstrument object.

The first score translation table shown in FIG. 34A is the same as thefirst score translation table shown in FIG. 27A. The second scoretranslation table shown in FIG. 34B is the same as the second scoretranslation table shown in FIG. 27B.

The fourth score translation table illustrates, for example, branchdistances of blood vessel objects through which the instrument object islikely to pass in the future. The third score translation tableillustrates branch distances of blood vessel objects through which theinstrument object has already passed. The fifth score translation tableillustrates branch distances of blood vessel objects that are unrelatedto the passage of the instrument object.

The adder 305 e refers to the score translation table 305 d to obtain ascore for each point on the blood vessel object, using a result of thedetermination in step S231 of FIG. 26, a spatial distance calculated instep S232, the branch distance calculated by the blood vessel branchdistance calculation unit 305 c, and the classification informationbased on a passage status of the instrument object.

As FIG. 34D shows, in the fourth score translation table of the bloodvessel objects through which the instrument object is likely to pass inthe future, relatively high scores are associated with the branchdistances. In contrast, as shown in FIG. 34C, in the third scoretranslation table of the blood vessel objects through which theinstrument object has already passed, lower scores than those in thefourth score translation table are associated with the branch distances.Also as shown in FIG. 34E, in the fifth score translation table of theblood vessel objects unrelated to the passage of the instrument object,lower scores than those in the third and fourth score translation tablesare associated with the branch distances. For example, by obtaining ascore associated with a branch distance using such a score translationtable, the adder 305 e can decrement scores of the branch distances ofblood vessel objects through which the leading end of the instrumentobject has already passed, and the blood vessel objects unrelated to thepassage of the instrument object, and increment scores of the branchdistances of blood vessel objects through which the instrument object islikely to pass in the future.

As described above, according to the three-dimensional display device300 of the present embodiment, the three-dimensional image can becorrected so as to suppress display of a blood vessel object throughwhich the instrument object has already passed or display of a bloodvessel object through which the instrument object does not pass. Thus,the three-dimensional display device 300 can prioritize display of ablood vessel object into which the instrument object is likely to beadvanced, thereby displaying a useful three-dimensional image.

In the present embodiment, the instrument traveling directiondetermination unit 304 identifies the blood vessel object through whichthe leading end of the instrument object has already passed, the bloodvessel object through which the leading end of the instrument object islikely to pass in the future, and the blood vessel object unrelated tothe passage of the leading end of the instrument object. However, theinstrument traveling direction determination unit 304 may not identifythese three types of blood vessel objects. In other words, theinstrument traveling direction determination unit 304 may identify atleast one of the blood vessel object through which the leading end ofthe instrument object has already passed, the blood vessel objectthrough which the leading end of the instrument object is likely to passin the future, and the blood vessel object that is unrelated to thepassage of the leading end of the instrument object.

Moreover, in the present embodiment, the three-dimensional displaydevice does not correct an object in the fusional area. However, anenhancement process may be performed on a blood vessel object that is ofparticularly high importance in the fusional area. The enhancementprocess can be implemented by, for example, contrast enhancement, edgeenhancement, or changing the display color.

This allows the three-dimensional display device to prioritize orenhance display of an image of an area of a blood vessel object in adirection of travel of the medical instrument in displaying athree-dimensional image of the blood vessel having the medicalinstrument, such as a catheter, inserted therein. Furthermore, thethree-dimensional display device can downgrade a priority level of anarea of the blood vessel through which the medical instrument hasalready passed, to process the image. This allows the three-dimensionaldisplay device to not only display merely a position of the blood vesselobject on the three-dimensional display but also more naturally andclearly display a blood vessel object which the viewer needs to see.

Embodiment 5

FIG. 35 is a block diagram of a functional configuration of athree-dimensional display device 500 according to an embodiment 5.

The three-dimensional display device 500 includes a gaze point obtainingunit 510, a fusional area determination unit 520, a blood vesselconnection information obtaining unit 530, a blood vessel importancecalculation unit 540, and a correction unit 550.

The gaze point obtaining unit 510 obtains a point of gaze of a viewer.The gaze point obtaining unit 510 is input means represented by, forexample, a pointing device such as a mouse and a touch panel. The viewerdesignates a point of gaze of the viewer in a displayedthree-dimensional image. The gaze point obtaining unit 510 obtains thepoint designated by the viewer in the image, as the point of gaze.Moreover, the gaze point obtaining unit 510 may obtain the point of gazeby, for example, detecting a viewing direction of the viewer. It shouldbe noted that the gaze point obtaining unit 510 may obtain the point ofgaze by a method other than the above.

The fusional area determination unit 520 determines a fusional area,based on the obtained position of the point of gaze. The fusional areais an area where binocular fusion is allowed. Specifically, the fusionalarea determination unit 520 stores fusional area information whichindicates, for example, positions of a plurality of points of gaze inthe depth direction, and a plurality of fusional areas respectivelycorresponding to the positions of the plurality of points of gaze in thedepth direction. In this case, the fusional area determination unit 520refers to the fusional area information to determine a fusional areathat corresponds to the obtained position of the point of gaze.

The blood vessel connection information obtaining unit 530, for example,obtains connection information of blood vessel objects in thethree-dimensional image from a storage unit (not shown) storing bloodvessel connection information or a recognition unit (not shown) whichgenerates blood vessel connection information from the three-dimensionalimage.

The blood vessel importance calculation unit 540 calculates theimportance of each of the blood vessel objects in the three-dimensionalimage, based on the fusional area determined by the fusional areadetermination unit 520, the blood vessel connection information obtainedby the blood vessel connection information obtaining unit 530, and depthinformation of the blood vessel object.

The correction unit 550 corrects an image which is displayed accordingto the importance levels of the blood vessel objects output by the bloodvessel importance calculation unit 540.

FIG. 36 is a flowchart illustrating processing operation of thethree-dimensional display device 500 according to the embodiment 5. Inaccordance with, for example, a display start indication from the viewerinput by display start input means not shown, the three-dimensionaldisplay device 500 starts the following processing:

First, the gaze point obtaining unit 510 obtains the three-dimensionalimage coordinates of a position of the point of gaze of the viewer inthe three-dimensional image (S510). The blood vessel connectioninformation obtaining unit 530 identifies the blood vessel object whichthe viewer is gazing, using the coordinates indicating the position ofthe point of gaze of the viewer obtained in step S510 and the currentlydisplayed three-dimensional image (S520). Furthermore, the blood vesselconnection information obtaining unit 530 obtains connection informationindicating connectivity of the blood vessel object which the viewer isgazing to each of the blood vessel objects included in thethree-dimensional image (S530). The blood vessel importance calculationunit 540 calculates the importance of each of the plurality of bloodvessel objects included in the three-dimensional image, based on theconnection information obtained in step S530 and a predeterminedfusional area (S540). The correction unit 550 segments thethree-dimensional image into a plurality of areas, based on theimportance levels calculated in step S540 (S550). The correction unit550 corrects the three-dimensional image for each area (S560).

As described above, according to the three-dimensional display device500 of the present embodiment, the visibility of a blood vessel objectthat not only has a small spatial distance to a blood vessel objectwhich the viewer is gazing but also has high connectivity to the bloodvessel object which the viewer is gazing can be improved. Thus, thevisibility of a blood vessel object that is useful to the viewer can beimproved. For example, the three-dimensional display device 500 canavoid the loss of information on a blood vessel object that has,although located outside the fusional area, a large connection with ablood vessel object which the use is gazing.

While in the present embodiment, the blood vessel importance calculationunit 540 calculates the importance of a blood vessel object based on theblood vessel connection information and the fusional area, it should benoted that the blood vessel importance calculation unit 540 maycalculate the importance of a blood vessel object so that the bloodvessel object, if included in the fusional area, is of higher importancethan if the blood vessel object is not included in the fusional area.This allows the three-dimensional display device 500 to display anatural image in which the blood vessel in the fusional area is lesslikely to be lost.

Embodiment 6

FIG. 37 is a block diagram of a functional configuration of athree-dimensional image processing device 600 according to an embodiment6.

The three-dimensional image processing device 600 includes a gaze pointobtaining unit 510, a fusional area determination unit 520, anidentification unit 610, and a correction unit 650.

The gaze point obtaining unit 510 obtains a point of gaze of a viewer.The gaze point obtaining unit 510 obtains a point in a displayedthree-dimensional image as the point of gaze of the viewer. Examples ofthe method to obtain the point of gaze in the image include a method toobtain the point of gaze based on a viewer's, manipulation such asdesignating the point by a pointing device or designating the point bydetecting a viewing direction, and a method to obtain a central point ona screen as the point of gaze. Moreover, the examples include a methodto obtain a leading end of a medical instrument identified by theidentification unit 610 as the point of gaze. The method to obtain thepoint of gaze may be other than the above.

The fusional area determination unit 520 determines a fusional areawhere binocular fusion is allowed, based on the obtained position of thepoint of gaze.

The identification unit 610 determines a relationship of the instrument,object with each, of the blood vessel objects, based on a position ofthe leading end of the instrument object in a displayedthree-dimensional image. Specifically, the identification unit 610identifies, among a plurality of blood vessel objects, a blood vesselobject through which the instrument object has already passed or a bloodvessel object through which the instrument object does not pass.

The correction unit 650 obtains the output of the fusional areadetermination unit 520 and the output of the identification unit 610.The correction unit 650 corrects the image so as to suppress display ofblood vessel objects outside the fusional area and, additionally,suppress display of the blood vessel object through which the instrumentobject has already passed that is identified by the identification unit610 in the displayed three-dimensional image.

As described above, according to the three-dimensional image processingdevice 600 of the present embodiment, the image can be corrected notonly so as to improve the visibility of a blood vessel object that has aclose spatial distance to the point of gaze of the viewer, but also soas to suppress display of a blood vessel object through which themedical instrument, such as a catheter, has already passed and improvethe visibility of the blood vessel object through which the medicalinstrument is likely to pass in the future.

Other Embodiment

While the three-dimensional display device according to only one or moreaspects has been described with reference to the exemplary embodiments,the present disclosure is not limited to the embodiments. Variousmodifications to the embodiments that may be conceived by those skilledin the art and combinations of components of different embodiments areintended to be included within the scope of the one or more aspects,without departing from the spirit of the present disclosure.

For example, while in the above embodiments 2 to 4, thethree-dimensional display device performs the series of operations whenviewpoint operation is input to the viewpoint input unit 102, thethree-dimensional display device may perform the series of operations ifa movement of the point of gaze by a certain amount or greater is input.This allows the three-dimensional image to be corrected appropriately inaccordance with the movement of the point of gaze, and the visibility ofthe three-dimensional image to be improved without requiring the viewerto input operation.

It should be noted that each component in each embodiment may take theform as dedicated hardware or may be implemented by executing a softwareprogram suitable for each component. The component may be implemented bya program execution unit, such as or CPU or processor, loading andexecuting the software program stored in a recording medium such as ahard disk or a semiconductor memory. Here, the software program forimplementing the three-dimensional display device according to the aboveembodiments is the following program.

Specifically, the program causes a computer to execute athree-dimensional display method including: obtaining a position of agaze point of a viewer; determining a fusional area where binocularfusion is allowed, based on the obtained position of the gaze point;correcting the three-dimensional image so as to suppress display of anobject which is included in the three-dimensional image outside thefusional area; and displaying the corrected three-dimensional image.

The herein disclosed subject matter is to be considered descriptive andillustrative only, and the appended Claims are of a scope intended tocover and encompass not only the particular embodiments disclosed, butalso equivalent structures, methods, and/or uses.

INDUSTRIAL APPLICABILITY

A three-dimensional image device according to one or more exemplaryembodiments disclosed herein is useful as a display device fordisplaying a three-dimensional image or a three-dimensional video, and,in particular, useful for televisions, computers, game machines, and soon.

The invention claimed is:
 1. A three-dimensional display device fordisplaying a three-dimensional image, comprising: a gaze point obtainingunit configured to obtain a position of a gaze point of a viewer; afusional area determination unit configured to determine a fusional areawhere binocular fusion is allowed, based on the obtained position of thegaze point; a correction unit configured to correct thethree-dimensional image so as to suppress display of an object which isincluded in the three-dimensional image outside the fusional area; adisplay unit configured to display the corrected three-dimensionalimage; and a fusional area information storage unit configured to storefusional area information which indicates positions of a plurality ofgaze points in a depth direction of the three-dimensional image and aplurality of fusional areas corresponding to the positions of theplurality of gaze points in the depth direction, wherein the fusionalarea determination unit is configured to refer to the fusional areainformation to determine the fusional area that corresponds to theobtained position of the gaze point.
 2. The three-dimensional displaydevice according to claim 1, wherein the correction unit is configuredto correct the three-dimensional image by removing an object which isincluded in the three-dimensional image and located on a side closer tothe viewer than the fusional area is.
 3. The three-dimensional displaydevice according to claim 1, wherein the correction unit is configuredto correct the three-dimensional image by blurring an object which isincluded in the three-dimensional image and located on a side fartheraway from the viewer than the fusional area is.
 4. The three-dimensionaldisplay device according to claim 1, further comprising a viewpointchanging unit configured to change a viewpoint of the three-dimensionalimage so that a display position of an object which is included in thethree-dimensional image and located at the position of the gaze pointdoes not change in a depth direction, wherein the correction unit isconfigured to correct the three-dimensional image the viewpoint of whichhas been changed.
 5. The three-dimensional display device according toclaim 4, wherein changing the viewpoint is a process of rotating thethree-dimensional image about the position of the gaze point.
 6. Athree-dimensional display device for displaying a three-dimensionalimage, comprising: a gaze point obtaining unit configured to obtain aposition of a gaze point of a viewer; a fusional area determination unitconfigured to determine a fusional area where binocular fusion isallowed, based on the obtained position of the gaze point; a correctionunit configured to correct the three-dimensional image so as to suppressdisplay of an object which is included in the three-dimensional imageoutside the fusional area; and a display unit configured to display thecorrected three-dimensional image, wherein the three-dimensional imageincludes a plurality of blood vessel objects representing a plurality ofblood vessels, three-dimensional display device further comprising: ablood vessel connection information obtaining unit configured to obtainconnection information indicating connectivity of a blood vessel objectlocated at the gaze point to each of the blood vessel objects includedin the three-dimensional image; and a blood vessel importancecalculation unit configured to calculate importance of each of the bloodvessel objects included in the three-dimensional image, based on thefusional area and the connection information, wherein the correctionunit is configured to correct the three-dimensional image so thatdisplay of a blood vessel object the importance of which is lower issuppressed to a greater extent.
 7. The three-dimensional display deviceaccording to claim 6, wherein the blood vessel importance calculationunit is configured to calculate, for each of the blood vessel objects,the importance of the blood vessel object so that the blood vesselobject, if included in the fusional area, is of higher importance thanif the blood vessel object is not included in the fusional area.
 8. Thethree-dimensional display device according to claim 6, wherein the bloodvessel importance calculation unit is configured to calculate, for eachof the blood vessel objects, the importance of the blood vessel objectso that the blood vessel object having a less number of blood vesselbranches to the blood vessel object located at the gaze point is ofhigher importance.
 9. The three-dimensional display device according toclaim 6, wherein the blood vessel importance calculation unit isconfigured to calculate, for each of the blood vessel objects, theimportance of the blood vessel objects so that the blood vessel objecthaving a smaller spatial distance to the blood vessel object located atthe gaze point is of higher importance.
 10. A three-dimensional displaydevice for displaying a three-dimensional image, comprising: a gazepoint obtaining unit configured to obtain a position of a gaze point ofa viewer; a fusional area determination unit configured to determine afusional area where binocular fusion is allowed, based on the obtainedposition of the gaze point; a correction unit configured to correct thethree-dimensional image so as to suppress display of an object which isincluded in the three-dimensional image outside the fusional area; and adisplay unit configured to display the corrected three-dimensionalimage, wherein the three-dimensional image includes a plurality of bloodvessel objects representing a plurality of blood vessels, and aninstrument object representing a medical instrument which is advancedthrough at least one of the blood vessel objects, the three-dimensionaldisplay device further comprising an identification unit configured toidentify, among the plurality of blood vessel objects, a blood vesselobject through which the instrument object has already passed or a bloodvessel object through which the instrument object does not pass, whereinthe correction unit is configured to correct the three-dimensional imageso as to suppress display of a blood vessel object which is locatedoutside the fusional area and through which the instrument object hasalready passed or display of a blood vessel object which is locatedoutside the fusional area and through which the instrument object doesnot pass.
 11. A three-dimensional display method for displaying athree-dimensional image, comprising: obtaining a position of a gazepoint of a viewer; determining a fusional area where binocular fusion isallowed, based on the obtained position of the gaze point; correctingthe three-dimensional image so as to suppress display of an object whichis included in the three-dimensional image outside the fusional area;and displaying the corrected three-dimensional image, wherein thethree-dimensional image includes a plurality of blood vessel objectsrepresenting a plurality of blood vessels, and an instrument objectrepresenting a medical instrument which is advanced through at least oneof the blood vessel objects, the three-dimensional display methodfurther comprising identifying, among the plurality of blood vesselobjects, a blood vessel object through which the instrument object hasalready passed or a blood vessel object through which the instrumentobject does not pass, wherein said correcting comprises correcting thethree-dimensional image so as to suppress display of a blood vesselobject which is located outside the fusional area and through which theinstrument object has already passed or display of a blood vessel objectwhich is located outside the fusional area and through which theinstrument object does not pass.
 12. A non-transitory computer-readablerecording medium storing a program for causing a computer to execute thethree-dimensional display method according to claim
 11. 13. Athree-dimensional image processing device for processing athree-dimensional image including a plurality of blood vessel objectsrepresenting a plurality of blood vessels, and an instrument objectrepresenting a medical instrument which is advanced through at least oneof the blood vessel objects, the three-dimensional image processingdevice comprising: a gaze point obtaining unit configured to obtain aposition of a gaze point of a viewer; a fusional area determination unitconfigured to determine a fusional area where binocular fusion isallowed, based on the obtained position of the gaze point; anidentification unit configured to identify, among the plurality of bloodvessel objects, a blood vessel object through which the instrumentobject has already passed or a blood vessel object through which theinstrument object does not pass; and a correction unit configured tocorrect the three-dimensional image so as to suppress display of a bloodvessel object which is located outside the fusional area and throughwhich the instrument object has already passed or display of a bloodvessel object which is located outside the fusional area and throughwhich the instrument object does not pass.